We propose a novel online learning paradigm for nonlinear-function estimationtasks based on the iterative projections in the L2 space with probabilitymeasure reflecting the stochastic property of input signals. The proposedlearning algorithm exploits the reproducing kernel of the so-called dictionarysubspace, based on the fact that any finite-dimensional space of functions hasa reproducing kernel characterized by the Gram matrix. The L2-space geometryprovides the best decorrelation property in principle. The proposed learningparadigm is significantly different from the conventional kernel-based learningparadigm in two senses: (i) the whole space is not a reproducing kernel Hilbertspace and (ii) the minimum mean squared error estimator gives the bestapproximation of the desired nonlinear function in the dictionary subspace. Itpreserves efficiency in computing the inner product as well as in updating theGram matrix when the dictionary grows. Monotone approximation, asymptoticoptimality, and convergence of the proposed algorithm are analyzed based on thevariable-metric version of adaptive projected subgradient method. Numericalexamples show the efficacy of the proposed algorithm for real data over avariety of methods including the extended Kalman filter and many batchmachine-learning methods such as the multilayer perceptron.
展开▼
